Stacking with an Extended Set of Meta-level Attributes and MLR

We propose a new set of meta-level features to be used for learning how to combine classifier predictions with stacking. This set includes the probability distributions predicted by the base-level classifiers and a combination of these with the certainty of the predictions. We use these features in conjunction with multi-response linear regression (MLR) at the meta-level. We empirically evaluate the proposed approach in comparison to several state-of-the-art methods for constructing ensembles of heterogeneous classifiers with stacking. Our approach performs better than existing stacking approaches and also better than selecting the best classifier from the ensemble by cross validation (unlike existing stacking approaches, which at best perform comparably to it).